Journals
  Publication Years
  Keywords
Search within results Open Search
Please wait a minute...
For Selected: Toggle Thumbnails
Neural Machine Translation Based on XLM-R Cross-lingual Pre-training Language Model
WANG Qian, LI Maoxi, WU Shuixiu, WANG Mingwen
Acta Scientiarum Naturalium Universitatis Pekinensis    2022, 58 (1): 29-36.   DOI: 10.13209/j.0479-8023.2021.109
Abstract659)   HTML    PDF(pc) (569KB)(179)       Save
The authors explore the application of XLM-R cross-lingual pre-training language model into the source language, into the target language and into both of them to improve the quality of machine translation, and propose three neural network models, which integrate pre-trained XLM-R multilingual word representation into the Transformer encoder, into the Transformer decoder and into both of them respectively. The experimental results on WMT English-German, IWSLT English-Portuguese and English-Vietnamese machine translation benchmarks show that integrating XLM-R model into Transformer encoder can effectively encode the source sentences and improve the system performance for resource-rich translation task. For resource-poor translation task, integrating XLM-R model can not only encode the source sentences well, but also supplement the source language knowledge and target language knowledge at the same time, thus improve the translation quality.
Related Articles | Metrics | Comments0
Neural Post-Editing Based on Machine Translation Quality Estimation
TAN Yiming, WANG Mingwen, LI Maoxi
Acta Scientiarum Naturalium Universitatis Pekinensis    2018, 54 (2): 255-261.   DOI: 10.13209/j.0479-8023.2017.153
Abstract1238)   HTML3)    PDF(pc) (629KB)(410)       Save

In order to solve the problem of overcorrection in automatic post-editing translations, the authors propose to make advantage of the neural post-editing (NPE) to build two special models: one is used to provide minor edit operations, the other is used to provide single edit operation, and make advantage of machine translation quality estimation to establish a filtering algorithm to integrate the special models with the regular NPE model into a jointed model. Experimental results on the test set of WMT16 APE shared task show that the proposed approach statistically outperforms the baseline. Deep analysis further confirms that proposed approach can bring considerable relief from the over-editing problem in APE.

Related Articles | Metrics | Comments0
Positional Language Models with Semantic Information
YU Wei,WANG Mingwen,WAN Jianyi,ZUO Jiali
Acta Scientiarum Naturalium Universitatis Pekinensis   
Abstract724)      PDF(pc) (558KB)(394)       Save
Because positional language models did not consider semantic relationship between the words in different positions, the authors present an effective model named “positional language models with semantic information”. Firstly, the authors use Gaussian kernel function to measure the position relationship between words. Secondly, the authors present a technology which is named “smoothed mutual information” to measure semantic relationship between the words, and also prove that smoothed mutual information can effectively solve the problem that a large number of two words could not calculate the transition probability between them only by mutual information. Then the authors prove that positional language models are a special case of positional language models with semantic information. Finally, applying this new model to the area of information retrieval can obtain a retrieval model based on the new model. The experiment show that the retrieval model based on the new model performs better than a retrieval model based on positional language models for using in information retrieval.
Related Articles | Metrics | Comments0